Thermodynamic non-additivity in disordered systems 
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It is shown that there is a mapping of the replica approach to disordered systems with 
finite replica index n on the Tsallis non-extensive statistics, if the average thermodynamic 
entropy differs from the information entropy for the probability distribution. In the case 
of incomplete information the entropic index q = 1 — n is shown to be related to the 
degree of lost information. 



I. INTRODUCTION 

The non-extensive thermodynamics results from 
a generalization [1] of the standard Gibbs- 
Boltzmann (GB) statistics. The non-extensive for- 
malism has been applied to the description of a va- 
riety of physical systems [2]. In some cases, related 
to disordered systems (e.g. anomalous diffusion in 
porous media or surface growth [3,4]), it has re- 
sulted in quantitative implications, which are diffi- 
cult to obtain in the framework of the classical GB 
approach. 

In the spirit of the information theory approach 
[5] the non-extensivity is usually introduced start- 
ing with a postulation of the entropy measure. For 
instance, Tsallis proposed [1] the following form 
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where p(cr) is a probability to find a state a, with 
J(da)p(a) = 1. Here q is an index, correspond- 
ing to a given statistics. Simultaneously q is a 
measure of the thermodynamic non-additivity. Al- 
though in general the non-extensivity should be 
distinguished [6] from the non-additivity, quite of- 
ten these two terms are being used as synonymous. 
If the state a is characterized by the energy e(a), 
then, based on the entropy maximization under 
a constraint for a generalized internal energy, one 
obtains [7,8] a power law for the probability 

!{do)[l-{l-q)[5e{a)Y^-<D [> 

We do not discuss here how the above distribu- 
tion modifies with different constraints [8] for the 
internal energy as well as the internal specifici- 
ties [8] of the Tsallis theory and its mathemat- 
ical consistency. For any constraint [8] in the 
limit q — > 1 one recovers the standard GB results: 
Sgb = — J (dcr)p(a) In p(a) and the exponential 
form for p(a). Note that the Tsallis form is not 
unique in having Sgb as a limit. As an alterna- 
tive example, one can consider the Renyi form [9], 
which is however additive. 

Despite of many efforts [10,11] towards deriv- 
ing the Tsallis-type entropy starting from basic as- 
sumptions, up to now it is not clear which physical 



systems are "naturally" non-extensive, and thus 
require the Tsallis-type statistics as an essential 
tool. In particular, the physical meaning of the q 
parameter [12 15] as well as a relation of to 
an underlying dynamics [16] are still under discus- 
sion. Also the thermodynamic consistency of the 
non-extensive formalism has recently been criti- 
cized [17]. Therefore, a derivation of the power-law 
entropy (like Sj) instead of postulating it, would 
be much helpful in resolving these problems. 

In this paper we investigate the conditions and 
assumptions at which the non-extensive thermo- 
dynamics can be recovered using the tools of the 
standard statistical mechanics, applied to systems 
which, because of their complexity, require more 
than the usual Hamiltonian description. This is in 
the spirit of the Beck-Cohen approach [18], who 
have shown that a non-GB distribution (including 
the Tsallis one) could appear in systems with fluc- 
tuating intensive quantities. Our purpose is to re- 
cover the Tsallis-type entropy, which is a generat- 
ing functional for the power-like probability distri- 
bution, but not to construct the distribution itself. 
This establishes a relation between the information 
theory and the statistical mechanics in application 
to systems which are out of the conventional GB 
equilibrium. In particular we focus on disordered 
systems since their description involves two essen- 
tial ingredients: a Hamiltonian and a distribution 
function. In fact, Tsallis has already commented 
[19] on a formal similarity of the non-extensive for- 
malism and the replica trick, suitable for the sys- 
tems with a quenched disorder. Therefore it is 
tempting to realize the condition at which such 
systems exhibit the non-extensive behavior. This 
would allow us to make a link between the system 
dynamics (the Hamiltonian), its complexity (the 
probability distribution) and the magnitude of the 
entropic q-parameter. 



II. REPLICA APPROACH TO 
DISORDERED SYSTEMS 



From a dynamical point of view disordered sys- 
tems (like porous materials [20] or spin glasses [21]) 



can be viewed as those consisting of many subsys- 
tems, whose equilibration times are much differ- 
ent [22]. Then, considering the static (or "equilib- 
rium") properties, the slower subsystem, say {cr}, 
is quenched with a given probability distribution 
P(cr), while the faster one, say {s} is governed by 
a Hamiltonian H = H R + H[a,s]. In the case of 
spin glasses the counterparts of {s} and {cr} sys- 
tems are the spin variables and the random fields 
(or exchange constants), respectively. It is essen- 
tial that the Hamiltonian contains a reference part 
H Rl which gives a characteristic energy scale re- 
flecting the way of quenching (e.g. the cooling rate 
and the correspondent temperature gap). 

For a given configuration {a} one can calculate 
the partition function 



Z[p) = Z/Z R = I (ds)t 
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where Z R = e - fJHR and /3 = 1/kT. Then all the 
thermodynamic characteristics are known. For in- 
stance, the free energy excess 



F(a)=F-H R = ~lnZ{a) 
and the internal energy excess 



(4) 



U(a)=U-H R = -—\nZ(a) (5) 



determine the thermodynamic entropy 
S t (<t) = P[U(ct) - F(<t)} 



(6) 



Then all the relevant thermodynamic quantities 
can be obtained by averaging the quenched ones. 
For instance, the average free energy is given by 



F = F(a) = J (da)P(a)F(a) 
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Note that, in contrast to the conventional equilib- 
rium, now we have two entropies - the thermody- 
namic entropy 



S T = / (da)P(a)S T {cr) 
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and the one related to the information on the prob- 
ability distribution 



Si = - J (da)P(a) In P(a) 



(9) 



Instead of the direct averaging of the logarithm 
in (7) it is convenient to introduce the replica trick 
[23], which is based on the representation 



InZ(a)=Iim [Z{ ° )]n - 1 



n— >0 n 



(10) 



It is equivalent to making n noninteracting copies 
of the system. Nevertheless the copies are not com- 
pletely independent because the probability distri- 
bution P(a) is the same for all of them. Then the 
problem reduces to the evaluation of the moments 
[Z(er)] n , which is expected to be a simpler task 
[24]. Note, however, that such a procedure is not 
just a trick. In fact, the behavior of one system 
(e.g. a spin glass) is interpreted as a limiting case 
of another system (the one with n ^ 0). Such 
that the quenched average is expressed through 
a "weighted" annealed average for a more com- 
plex system, involving an extension of the phase 
space [25]. The systems with n ^ were consid- 
ered [26-28] in application to the spin glasses and 
neural networks [29] . In the context of spin models 
such a system can be viewed as a collection of n 
domains of interacting spins with a common prob- 
ability distribution P(<r). Note, however that due 
to the limiting procedure in eq. (10) the replica 
index n can be non-integer [24]- i.e. at least for 
< n < 1 it must vary continuously. Sherrington 
[26] has investigated the critical behavior of such 
replica magnets, driven by the inter- and intra- 
rcplica interaction. Moreover, Derrida has shown 
[30], that there is a mapping between the random 
walk, the random energy model and the replica re- 
sults for non-zero n. 

Based on the discussion above we allow n to be 
finite. The cr-dependent functions, like F(p), obey 
the standard statistical thermodynamics, since a 
is a parameter (although known only statistically 
at the level of F[p) evaluation). The average free 
energy is now n-dependent 

F n = JUa) =~f (da)P(a) [Z{a)] ^~ 1 (11) 

The internal energy U n can be found as a coun- 
terpart of the internal energy (5) and the average 
internal energy 

Un = ~ ^{do)P{o)\Z{o)X n - x dZ{a)ld{3 (12) 

can be rearranged to the form 

U n - J (da)P(a)[Z(a)] n U(a) (13) 

If cr-dependent quantities obey the standard ther- 
modynamic relation F{a) = U{a) — St(ct) / /3, then 
the counterpart of the entropy is given by 



S n = / (da)P(a) 
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(14) 



III. MAPPING ON THE TSALLIS THEORY 



(23) 



Starting from eq. (14) and replacing n by 1 — q 
we can rearrange the correspondent thermody- 
namic entropy S q to the Tsallis construction 



l-J(da)[U(a)}" 
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where n(cr) is given by 

11(a) = [P(a)[Z(a)Y-«{l + (1 - q)pU(a)}} 1/q 

(16) 

At this level our calculations are purely formal, 
and S q is equivalent to S n . In order to obtain 
S q coherently with the Tsallis conjecture we have 
to associate n(er) with a probability distribution. 
Then we naturally impose two restrictions: 

n(cr) > 0; J (da)U(a) = 1 (17) 



From the positivity condition we have 
1 



q < 1 + 



(3U(a) 
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which is equivalent to the cut-off condition [8] in 
the Tsallis approach. 

In order to realize the meaning of q ^ 1 we ex- 
pand II(cr), given by eq. (16), around q = 1 up to 
the second order 



U(a) = P(a)-P(a)D(a)(q-l) + 



P(a) 
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where D(a) = St(c) + lnP(<j). Therefore, we see 
that the probability distribution differs from P(cr) 
as long as q ^ 1. Moreover, the magnitude of q is 
coupled to the cr-dependent thermodynamics and 
to a shape of P(cr). 

From the normalization condition we obtain 



suggesting that small deviations of q from 1 are 
determined by the difference AS of the thermo- 
dynamic and the information entropies. Com- 
paring eqs.(8) and (9) we can easily demonstrate 
that these two entropic impacts are equal only 
if the probability distribution is consistent with 
the thermodynamic fluctuations [31]: P{cr) oc 
exp(— (3St{<j)). Therefore, AS (and consequently 
q) can be viewed as a measure of the deviation 
from the conventional thermodynamic equilibrium. 
Similar conjectures were drawn in ref. [32], dis- 
cussing the fluctuations within the Tsallis statis- 
tics. Since the sign of AS* and E is not fixed, then 
from eq. (23) we can have both q > 1 and q < 1. 
Nevertheless, q is related to the replica index n 
through n = 1 — q. Therefore, if n > (as it is 
usually assumed in the spin glass theories), then 
we deal with q < 1. 

On the other hand AS depends explicitly on the 
thermodynamic parameters and on a shape of the 
quenching distribution. Therefore, there is a map- 
ping of the replica approach with a non-zero replica 
index n = 1 — q onto the Tsallis theory, provided 
that the replica index varies coherently with the 
system thermodynamics and the probability dis- 
tribution. This conclusion agrees with the results 
obtained within the incomplete information theory 
[33] which suggests that the q parameter is related 
to the thermodynamic quantities. In the case when 
the thermodynamic parameters are fixed, the mag- 
nitude of q is related to the parameters of the prob- 
ability distribution. If, for instance, P(a) is of the 
gaussian form, then the width of this distribution 
determines the fluctuation a 2 — a 2 . In other words 
the magnitude of q is related to the fluctuation of 
the parameters in the quenching distribution. In 
some sense, this result is coherent with the one [14] 
obtained for the generalized exponential distribu- 
tion. 



IV. INCOMPLETE INFORMATION 



-AS(q - 1) + [AS + E] [q - l) 2 = (20) 



where 



AS 



J (da)P(a)D(a 



Si 



(21) 



is the difference of the thermodynamic and the in- 
formation entropies and E is given by 

E = \ J (da)P(a) [D 2 {a) (3 2 U 2 {a)} (22) 

Then solving eq. (20) with respect to q we obtain 
the result 



As is discussed above, due to fact that (j ^ 1, 
we deal with the weighted annealed averaging (see 
eq. (14)). That is, in contrast to the usual quench- 
ing (q = 1) we have a nonvanishing feed-back ef- 
fect on {er}-distribution from the thermodynam- 
ics of {s}-subsystem (see eq. (19)). As a result 
we operate with the probability distribution n(cr), 
involving the thermodynamic quantities, instead 
of P{<j) which corresponds to the case when {er}- 
subsystem remains unchanged. This situation is 
similar to what is reported [28,29] for partially an- 
nealed systems. 

Taking the porous materials [20] as an example, 
we can associate P(cr) with the empty matrix (e.g. 



pore size distribution) and II (er) - with the matrix 
in the presence of an adsorbed fluid. In general, 
the latter changes the matrix properties. For in- 
stance, by the analogy with the intercalation sys- 
tem [34,35], we can expect a change of the matrix 
volume upon the fluid absorption. Moreover, quite 
often the adsorbates induce a new ordering [36] of 
the substrates or facilitate their disordering [37] 
depending upon the specificities of the adsorbate- 
substrate interaction. Therefore, because of a com- 
plicated fluid-matrix coupling, our information on 
the pore size distribution could be incomplete [33] 

J(da)P(a) = l; J (da)U(cr) = A^1 (24) 

where 1 — A can be considered as a degree of lost 
information. In this case we can rearrange (14) to 
the following form 



■J(da)U(a) 
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(25) 



where II(cr) should be found from (if q ^ 1) 



[U(aW n(a) = 
P(a) [[Z{a)Y-i{l + (1 - q)pU(a)} l] (26) 

For small 1 — A the distribution II (a) should ex- 
hibit small deviations from that given by eq. (16). 
Therefore, we determine 11(a) perturbatively, tak- 
ing eq. (16) as a zeroth approximation. Then, ex- 
panding around q = 1, we find that the linear ap- 
proximation is the same as that in eq. (19) 



U(a)=P(a)-P(a)D(a)(q-l) 



(27) 



From the incomplete normalization condition (24) 
we obtain 



1 + 



1- A 
~AS~ 



(28) 



which demonstrates that the entropic index q is 
related to the degree of lost information (1 — A). 
In the limit A — > 1 we must retain the quadratic 
term in (27) and then we recover our previous re- 
sult (23). 



V. CONCLUSION 

A relation between the non-extensive Tsallis 
statistics and the replica formalism for disordered 
systems is discussed. The key specificity of this 
kind of systems is the splitting into subsystems 
with different levels of description, i.e. the Hamil- 
tonian and the probability distribution. Due to 
this one necessarily operates with two entropic im- 
pacts: the thermodynamic and the information en- 
tropies. It is shown that there is a mapping of 



the replica approach to disordered systems on the 
Tsallis theory with q ^ 1, if the average thermo- 
dynamic entropy differs from the information en- 
tropy related to the probability distribution. In the 
case of incomplete information the entropic index 
deviates from unity proportionally to the degree 
of lost information. Therefore, the magnitude of 
q depends on the thermodynamic parameters as 
well as on a shape of the distribution. This con- 
clusion is coherent with the results of other studies 
[14,18,32,33] discussing the meaning of the q index 
in the Tsallis approach. 

Our results suggest that a similar mapping can 
be expected for other systems which are out of the 
thermodynamic equilibrium (at least in its tradi- 
tional meaning). Then the problem involves ad- 
ditional relevant quantities, related to a deviation 
from the conventional equilibrium. For instance, 
the deviation could be induced by the heat bath 
properties [15], by fluctuations of intensive ther- 
modynamic parameters [18], or by finite sizes (no 
thermodynamic limit) [38]. 

It should be emphasized that the non-additive 
nature of the system discussed here does not de- 
pend on the approximations made. From the every 
formulation of the problem we have two entropies 
Si and St- If we consider a "total" entropy f2 as 
a measure of our uncertainty on the system state, 
then it is clear that ft cannot be written as an ad- 
ditive combination of Si and St because these two 
are not independent. Therefore it would be inter- 
esting to find a way of solving this more general 
problem without introducung model arguments. 
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